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FOREWORD 


Opinions,  interpretations,  conclusions  and  reconmendations  are 
those  of  the  author  and  are  not  necessarily  endorsed  by  the  U.S. 
Army. 

Where  copyrighted  material  is  quoted,  permission  has  been 
obtained  to  use  such  material . 


X/4"  Where  material  from  documents  designated  for  limited 
distribution  is  quoted,  permission  has  been  obtained  to  use  the 
material . 

Citations  of  commercial  organizations  and  trade  names  in 
this  report  do  not  constitute  an  official  Department  of  Army 
endorsement  or  approval  of  the  products  or  services  of  these 
organizations . 

it  In  conducting  research  using  animals,  the  investigator (s) 
adhered  to  the  "Guide  for  the  Care  and  Use  of  Laboratory 
Animals,"  prepared  by  the  Committee  on  Care  and  use  of  Laboratory 
Animals  of  the  Institute  of  Laboratory  Resources,  national 
Research  Council  (NIH  Publication  No.  86-23,  Revised  1985). 


0 


For  the  protection  of  human  subjects,  the  investigator (s) 


adhered  to  policies  of  applicable  Federal  Law  45  CFR  46. 

iL  In  conducting  research  utilizing  recombinant  DNA  technology, 
the  investigator (s)  adhered  to  current  guidelines  promulgated  by 
the  National  Institutes  of  Health. 


In  the  conduct  of  research  utilizing  recombinant  DNA,  the 


investigator ( s )  adhered  to  the  NIH  Guidelines  for  Research 
Involving  Recombinant  DNA  Molecules . 

In  the  conduct  of  research  involving  hazardous  organisms, 
the  investigator (s)  adhered  to  the  CDC-NIH  Guide  for  Biosafety  in 
Microbiological  and  Biomedical  Laboratories . 
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Introduction 

Breast  cancer  is  one  of  the  leading  causes  of  death  among  women.  However,  there  is  clear 
evidence  that  early  diagnosis  and  subsequent  treatment  can  significantly  improve  the  chance 
of  survival  for  patients  with  breast  cancer. 

Mammography  has  become  one  of  the  major  diagnostic  procedures  with  a  proven  capability 
for  detecting  early-stage,  clinically  occult  breast  cancers.^-*  However,  breast  cancers  in  their 
early  stage  are  small  and  frequently  their  radiographic  appearance  differs  only  subtly  from 
that  of  normal  tissue  or  benign  abnormalities.  Because  of  this  subtlety,  the  potential  for 
misclassification  by  radiologists  is  substantial.  Only  10-30%  of  cases  that  have 
mammographically  suspicious  findings  and  are  subjected  to  biopsy  prove  to  be  malignant.^ 
On  the  other  hand,  approximately  10-30%  of  patients  with  breast  cancer  are  misdiagnosed  by 
mammography  (have  the  cancer  missed  or  not  detected  on  their  mammograms).^®"!^ 

Besides  the  subtle  nature  of  radiographic  lesions  associated  with  breast  cancer,  many  errors  in 
radiological  diagnoses  can  be  attributed  to  human  factors  such  as  subjective  or  varying 
decision  criteria,  distraction  by  other  image  features,  and  simple  oversight.  studies 
suggest  that  these  errors  may  occur  even  with  experienced  radiologists. These  errors 
may  be  reduced  by  the  use  of  automated  detection  schemes  that  can  locate  and  classify 
possible  lesions,  thereby  alerting  the  radiologist  to  examine  these  areas  with  particular 
caution.  Moreover,  the  automated  detection  schemes  can  serve  as  a  "second  radiologist", 
similar  to  the  double  reading  by  two  radiologists  that  is  commonly  practiced  in  diagnostic 
radiology  to  increase  diagnostic  efficacy. 

Microcalcifications  are  commonly  considered  to  be  important  signs  of  breast  cancer.  It  has 
been  reported  that  30-50%  of  breast  cancers  detected  radiographically  demonstrate 
microcalcifications  on  mammograms.70-25  Up  to  90%  of  cases  of  ductal  carcinoma  in  situ 
present  with  microcaleifications.26  The  correlation  between  the  presence  of 
microcalcifications  and  the  presence  of  breast  cancer  suggests  that  accurate  detection  of 
microcalcifications  will  improve  the  efficacy  of  mammography  as  a  diagnostic  procedure. 

Microcalcifieations  occur  in  malignant  and  benign  conditions.  Some  microcalcifications  are 
characteristically  benign  or  are  associated  with  a  benign  process.  For  example,  calcified 
fibroadenomas  have  a  typical  "popcorn"  configuration  appearing  coarse  and  solitary.  Milk  of 
calcium  demonstrates  sedimentation.^^  Vascular  calcifications  have  a  tram  track  appearance, 
typical  of  vascular  calcifications  seen  in  other  areas  of  the  body.  Dermal  calcifications  tend 
to  be  smooth  and  round  with  lucent  centers.  Secretory  calcifications  are  thick,  smooth,  cigar¬ 
shaped,  and  usually  non-branching.  Features  supporting  benignity  include  uniform  size  and 
density  of  the  calcium  flecks,  as  is  seen  in  sclerosing  adenosis.^*  Furthermore,  benign 
mierocaleifications  tend  to  be  uniformly  dense  or  scattered,  without  a  segmental  or  linear 

distribution.29 

Some  microcalcifications  associated  with  malignancy  have  a  typically  granular  or  linear 
appearance.  They  usually  oecur  in  clusters  consisting  of  greater  than  15  particles.^o  The 
particle  size  is  small  (less  than  1  mm)  and  the  shape  is  irregular.^i  Some  clusters  of 
mierocaleifications  have  neither  the  typically  benign  nor  typically  mahgnant  configurations 
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described  above.  These  "indeterminate"  microcalcifications  present  a  significant  diagnostic 
problem  and  require  careful  analysis. 

The  number  of  microcalcifications  per  cm^  has  been  shown  to  be  the  most  important 
predictor  of  malignancy,  with  clusters  consisting  of  less  than  10  microcalcifications  per  cm^ 
having  a  high  chance  of  benignity.  Clusters  consisting  of  microcalcifications  numbering 
greater  than  15  per  cm^  have  a  higher  chance  of  malignancy.32 

The  task  of  detection  and  classification  of  microcalcifications  for  the  diagnosis  of  breast 
cancer  is  a  difficult  one.  The  inability  to  correctly  predict  cancer  is  not  only  due  to  the 
overlap  in  appearance  between  microcalcifications  associated  with  benign  and  malignant 
conditions,  dense  breasts,  improper  technical  factors  or  simple  oversight  by  radiologists  may 
contribute  to  the  failure  to  detect  microcalcifications.  Differing  levels  of  confidence  and 
training  among  interpreting  radiologists  may  lead  to  inconsistent  recommendations  for 
management. 

Radiologists  classify  breast  microcalcifications  into  one  of  three  groups:  benign,  likely 
malignant,  and  indeterminate.  Most  patients  with  indeterminate  types  of  calcifications 
undergo  a  breast  biopsy  to  exclude  cancer.  Any  method  that  would  correctly  classify  benign 
types  of  calcifications  previously  considered  indeterminate  would  decrease  the  frequency  of 
biopsy  and  therefore  the  cost  of  detection  of  breast  cancer. 

Several  investigators  have  been  developing  computer  programs  for  the  automated  detection 
of  microcalcifications  on  mammograms.^^"^^  Chan  et  al.  showed  that  the  computer  program 
can  detect  subtle  microcalcifications  that  may  be  missed  by  radiologists,  indicating  that  it  is  a 
promising  approach  to  the  automated  detection  of  microcalcifications.^^  More  recently,  Wu 
et  al.  applied  an  artificial  neural  network  (ANN)  to  detect  microcalcifications.^^  The  ANN, 
trained  by  using  the  power  spectrum  of  regions  of  interests  (ROI)  containing 
microcalcifications,  was  able  to  eliminate  50%  of  false-positive  detections  of  a  rule-based 
scheme^^,  39  while  preserving  more  than  95%  of  the  true-positive  detections.  The  neural 
network  achieved  an  A^  value  of  0.85  for  the  detection  of  clustered  microcalcifications. 
Several  other  computer  schemes  for  detection  of  microcalcifications  were  also  reported  by 
Astley  et  al.,^  based  on  likelihood  estimators,  by  Grimaud  et  a/.,4i  using  mathematical 
morphology  tools,  and  by  Karssemeijer,'^^  using  a  stochastic  method  based  on  Bayesian 
decision  theory. 

As  stated  earlier,  microcalcifications  can  be  associated  with  either  benign  or  malignant 
processes.  It  is  important  to  distinguish  different  types  of  microcalcifications  after  they  have 
been  identified  by  a  detection  scheme.  Accurate  classification  of  microcalcifications  into 
benign  and  malignant  groups  would  help  improve  the  sensitivity  of  the  diagnosis  as  well  as 
reduce  the  number  of  imnecessary  biopsies. 

As  the  first  step  in  the  process  of  developing  an  automated  computer  scheme  for 
classification  of  microcalcifications,  a  neural  network  system  was  developed  to  classify 
microcalcifications  in  the  radiographs  of  biopsy  specimens.  Classification  of 
microcalcifications  in  radiographs  of  biopsy  specimens  is  an  “idealized”  situation. 
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In  biopsy  specimens,  underlying  tissue  around  microcalcifications  is  less  than  that  present  in 
normal  mammograms.  Therefore,  the  scatter  radiation  recorded  on  films  is  reduced,  resulting 
in  improved  contrast.  Higher  dose  and  geometrical  magnification  can  also  be  used  to  obtain 
radiographs  of  biopsy  specimens  as  compared  with  regular  mammograms.  Less 
magnification  results  in  less  geometrical  unsharpness.  Higher  exposure  can  be  used  to 
achieve  greater  signal  to  noise  ratio  and  thereby  improve  image  quality  of  radiographs. 

Therefore,  microcalcifications  in  the  radiographs  of  biopsy  specimens  are  more  clearly 
represented  than  those  in  regular  mammograms.  After  we  can  successfully  apply  our 
algorithm  to  classify  microcalcifications  in  radiographs  of  specimens,  we  will  make 
necessary  adjustments  to  apply  the  algorithm  to  the  regular  manunograms. 

In  recent  years,  rapid  progress  of  research  on  artificial  neural  networks  (ANN)''^^  has  been 
reported  extensively  in  the  field  of  computer  science  and  many  applied  fields.  Neural 
networks  address  detection,  classification,  and  decision-making  problems  not  by  pre¬ 
specified  "conventional"  algorithms,  but  rather  by  "learning"  from  examples  presented 
repeatedly.  The  popularity  of  neural  networks  is  primarily  due  to  their  apparent  ability  to 
make  decisions  and  draw  conclusions  when  presented  with  complex,  noisy,  or  partial 
information  and  to  adapt  their  behavior  to  the  properties  of  the  training  data.  Neural 
networks  are  capable  of  parallel-processing  a  large  amount  of  information  simultaneously 
and  have  been  shown^^,  45  to  be  a  useful  tool  for  pattern  recognition  in  fields  where 
conventional  algorithmic  approaches  and  rule-based  expert  systems  may  not  be  successful. 

Conventional  diagnosis  of  breast  cancer,  a  combination  of  mammography  and  physical 
examination  has  been  the  most  effective  screening  methods  to  date.  However,  the  ability  of 
the  conventional  techniques  to  discover  cancer  with  an  acceptable  reliability  is  limited.  The 
limitations  are  manifest  in  the  facts  that  average  positive  predicative  value  for  suspicious 
breast  lesions  is  only  20-30%  at  surgical  pathology  and  approximately  10-30%  of  breast 
cancer  are  missed  by  mammography,  i  4. 46  ^  addition,  manunography  may  fail  to  reveal 
breast  lesions  that  are  present  in  certain  groups  of  women  such  as  those  with 
mammographically  dense  breasts,  women  with  prior  breast  cancer  who  are  at  risk  for 
recurrence  following  therapy,  and  those  with  prosthetic  breast  implants. 

Recent  studies  have  suggested  that  BMRI  may  be  an  effective  tool  that  complements  the 
conventional  techniques  such  as  manunography  in  the  diagnosis  and  management  of  breast 
carcinoma.'^^'^®  has  a  very  high  sensitivity  in  detecting  cancer  lesions.  The 

applicability  of  MRI  as  a  diagnostic  procedure  in  breast  cancer  is  currently  limited,  however, 
by  the  low  specificity  and  high  cost.  While  BMRI  shows  promise  in  solving  some  of  the 
difficult  diagnostic  problems,  the  time  required  to  examine  the  massive  amounts  of  data 
resulting  from  each  patient  can  potentially  make  cost  of  MRI  very  high.  Technical  advances 
have  decreased  the  data  acquisition  time  to  approximately  fifteen  minutes  per  patient. 
However,  data  analysis  remains  cumbersome  at  best  and  a  stumbling  block  to  ready  clinical 
acceptance.  It  can  take  1-2  hours  for  a  radiologist  to  analyze  the  large  amount  of  images  of 
each  patient.  The  analysis  performed  often  have  qualitative  and  subjective  properties,  which 
is  one  of  the  reasons  for  the  low  specificity  of  BMRI. 
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We  have  developed  an  image  display  and  analysis  system  that  allows  interactive  3D  image 
manipulation  and  qualitative  analysis  of  selected  image  regions.  This  computer  visualization 
system  can  help  radiologists  improve  the  efficacy  of  examining  the  massive  amount  of  data. 
Tlie  computer  system  may  improve  the  specificity  of  detecting  breast  carcinoma,  reduce  the 
subjectivity  of  radiologists  in  interpreting  BMRI  images,  and  cut  the  cost  by  reducing  the 
time  radiologists  have  to  spend  reading  the  large  number  of  images.  This  system  can  make 
BMRI  a  cost-effect  procedure  with  high  sensitivity  and  specificity  in  the  diagnosis  of  breast 
cancer. 
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Body  of  Report 


METHODS 


Algorithms  for  classification  of  microcalcifications  have  been  developed  in  two  stages.  First, 
a  convolution  neural  network  based  method  was  developed  to  classify  microcalcifications 
using  radiographs  of  pathological  specimen.  Second,  an  image  feature  based  method  was 
developed  to  classify  microcalcifications  using  real  mammograms.  The  image  features  used 
and  optimal  structure  of  the  backpropagation  neural  network  were  determined  by  using 
genetic  algorithm. 

Classification  of  microcalcifications  in  pathological  specimen 

The  overall  approach  for  the  classification  of  microcalcifications  using  a  CNN  system  is 
shown  in  Figure  1.  The  radiographs  of  pathological  specimen  are  digitized  by  a  high 
resolution  digitizer.  Regions  of  interest  (ROI)  containing  microcalcifications  are  manually 
selected.  These  ROIs  are  preprocessed  and  used  as  input  to  the  CNN  system.  Finally,  the 
classification  results  are  examined  by  the  ROC  analysis. 


1  Biopsy  Specimen  | 

_ J 

1  Radiographs  | 

_ j 

1  Digitization  | 

_ i 

r 

1  Preprocessing  | 

r 

1  CNN  1 

1 

1  Diaj 

];nosis  | 

1 

1  ROC. 

Analysis  | 

Figure  1 .  Overall  approach  for  the  classification  of  microcalcifications  using  CNN. 
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The  classification  of  microcalcifications  is  based  primarily  on  the  fact  that 
microcalcifications  associated  with  malignant  processes  generally  have  more  irregular  shapes 
with  fiizzy  and  spiculated  boundaries  and  are  less  uniform  in  density  and  size.  They  are 
usually  grouped  into  multi-particle  clusters.  The  microcalcifications  associated  with  benign 
processes,  on  the  other  hand,  usually  tend  to  have  smoother  and  well-defined  boundaries, 
rounded  shape,  and  uniform  densities  and  sizes.  The  neural  network  system  will  be  trained  to 
recognize  the  characteristics  of  each  type  of  microcalcifications. 

The  CNN  is  based  on  the  network  stracture  of  Fukushima’s  Neocognitron^i  which  is 
designed  to  simulate  the  vision  of  vertebrate  animals.  The  structure  of  CNN  used  in  this 
study  resembles  a  simplified  Neocognitron.  A  two-dimensional  convolution  operation  from 
the  input  layer  to  the  hidden  layer  is  employed  to  simulate  radiologists’  viewing  of  a 
suspected  area.  The  CNN  has  the  ability  to  process  and  recognize  two  dimensional  image 
patterns  and  has  been  shown  to  be  an  effective  tool  in  image  processing  and  pattern 

i:ecognition.52-54 

Acquisition  of  Mammograms 

The  selected  radiographs  of  breast  biopsy  specimen  are  digitized  with  an  image  resolution  of 
21pm  X  21pm  per  pixel  by  a  CCD  camera  digitizer  (DBA  Systems  Inc.).  With  high 
resolution  digitization,  the  morphological  information  of  microcalcifications  can  be 
preserved,  which  enables  the  neural  network  system  to  differentiate  different  types  of 
microcalcifications  on  the  basis  of  their  geometrical  shapes  and  density  patterns.  Figure  2(a) 
shows  a  cluster  of  microcalcifications  in  original  radiographs  of  pathological  specimen. 
Shown  in  Figure  2(b)  are  clustered  microcalcifications  after  being  digitized  with  the  high 
resolution  digitizer.  The  shapes  and  density  patterns  of  the  microcalcifications  are  better 
defined  in  Fig.  2(b)  than  those  shown  in  Fig.  2(a). 


(a)  (b) 

Figure  2.  Microcalcifications  shown  in  an  original  radiograph  of  pathological  specimen 
(a)  and  shown  in  a  radiograph  digitized  with  a  high  resolution  (21  pm  x  21  pm) 
digitizer  (b). 
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Convolution  Neural  Network 

The  structure  of  the  CNN^^  is  shown  in  Figure  3.  The  input  to  the  CNN  are  ROIs  of  matrix 
size  of  64  x  64  pixels,  containing  benign  or  malignant  type  of  microcalcifications.  Only  one 
hidden  layer  is  used  in  this  study.  The  connections  between  input  and  hidden  layer  are 
grouped  into  seven  different  kernels  based  on  the  structure  of  Fukushima's  Neocognitron.^b 
55  There  are  two  output  units  in  the  output  layer,  with  each  unit  corresponding  to  a  benign  or 
malignant  class  of  microcalcifications.  The  hidden  layer  and  the  ouQiut  layers  are  folly 
connected. 


Input  Hidden  Output 


Figure  3.  Structure  of  a  convolution  neural  network  used  for  the  classification  of 
microcalcifications. 

The  training  algorithm  of  the  CNN  is  similar  to  that  of  a  baclqiropagation  neural  network,  in 
which  backpropagation  and  the  generalized  delta  rule  are  used  in  the  training  process.56  The 
input  signals  are  now  two  dimensional  images.  The  weights  are  all  arranged  in  the 
convolution  kernels.  In  the  feed-forward  propagation,  the  output  of  /th  layer  are  first 
convoluted  with  weight  filters.  The  sum  of  the  convolution  is  then  added  by  a  bias  term  to 
form  the  net  input  to  the  next  layer. 


p=i 

where  N‘p\x,y)  is  the  net  input  to  the  unit  (x,y)  in  layer  1+1 ,  0'p(x,y)  is  the  output  of  the 
unit  in  layer  I,  w‘p^gix,y)  is  a  weight  kernel,  and  is  a  bias  term  in  layer  1+1 .  hi  the 

notation,  layer  number  /  =  (1, 2, ...,  L),  cluster  number  in  the  /th  layer  p  =  (1, 2, ...,  P^),  and 
cluster  number  in  the  (/+i)th  layer  q  =  (1, 2, ...,  Note  that  *  denotes  discrete 

convolution. 
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[O',  = z  ^  j  -S'))  • 

We  can  then  rewrite  equation  [1]  as 


[2] 


‘  j 


0!;\x,y)=nN'"(x.y)), 


7+1/ 


[3] 


where /is  the  activation  function 


1 


1  +  exp(-jc) 


[4] 


hi  the  error  backpropagation,  the  weights  are  modified,  similar  to  that  in  Eqn.[2],  as  the 
following, 


to  minimize  the  error  function, 

E  =  \'E(.nx,y)-0^ix,y))\  [6] 

where  T(x,y)  is  the  target  output. 

In  the  training  process  of  the  CNN,  each  image  block  is  rotated  and  reflected  such  that  the 
number  of  training  data  are  increased  eight  fold.  The  rotation  and  reflection  represent 
different  orientations  of  microcalcifications  in  mammograms.  The  training  widi  additional 
orientation  can  effectively  make  the  CNN  rotational  invariant. 
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Classification  of  microcalcifications  in  real  mammograms 

An  alternative  approach  was  employed  to  classify  microcalcifications  of  in  clinical 
mammograms.  Microcalcifications  can  be  characterized  by  a  number  of  quantitative 
variables  describing  the  underlying  key  features  of  a  suspicious  region  such  as  the  size, 
shape,  and  number  of  microcalcifications  in  a  cluster.  These  features  are  calculated  by  an 
automated  extraction  scheme  for  each  of  the  selected  regions.  The  features  are  then  used  as 
input  to  a  baclqjropagation  nemal  network  to  make  a  decision  regarding  the  probability  of 
malignancy  of  a  selected  region.  The  initial  selection  of  image  features  set  was  a  rough 
estimation  that  may  include  redundant  and  non-discriminant  features.  A  genetic  algorithm 
was  employed  to  select  an  optimal  image  feature  set  from  the  initial  feature  set  and  select  an 
optimized  structure  of  the  neural  network  for  the  optimal  input  features.  Finally,  the 
performance  of  optimized  neural  network  is  evaluated  using  the  selected  sub-features.  The 
performance  of  neural  network  is  then  compared  with  that  of  radiologists  in  classifying  the 
clusters  of  microcalcifications. 

A  selected  region  is  a  256  x  256  matrix  from  a  mammogram  that  is  digitized  at  50  microns 
per  pixel  with  a  Lumisys  scanner.  Figure  4  shows  two  regions  of  interest  (ROI)  selected  from 
original  mammograms:  (a)  benign  and  (b)  malignant.  The  background  trend  in  a  selected 
region  is  eliminated  by  an  adaptive  trend  correction  technique.  The  trend-corrected  image  is 
then  binarized  by  taking  the  top  level  of  32  quantization  levels.  A  series  of  pre-processing 
techniques  that  include  open  and  dilation  operations,  logic  AND,  and  labeling  operation  are 
applied  to  the  binary  image  to  eliminate  artifacts  while  preserving  the  original  shapes  of 
microcalcifications.  Image  features  are  extracted  based  on  the  pre-processed  image  and  two 
morphologically  filtered  images.  Nineteen  image  features  are  extracted  for  each  cluster  of 
microcalcifications  characterizing  the  size,  number  of  microcalcification  in  the  cluster,  and 
shape. 


(a) 


(b) 


Figure  4  Two  ROIs  of  size  256  x  256  selected  from  original  mammgrams:  (a)  benign  and 
(b)  malignant 
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Pre-Processing 

The  pre-processing  algorithm  is  demonstrated  in  Figure  5.  The  purpose  of  pre-processing  is 
to  obtain  a  binary  image  of  microcalcifications.  The  pixels  representing  microcalcifications 
have  values  of  one  and  the  rest  have  values  of  zero.  We  need  only  binary  image  because 
image  features  we  use  are  not  based  on  density  but  rather  based  on  only  size,  shape, 
clustering,  and  number  of  microcalcifications  in  a  cluster. 


Figure  5  Pre-Processing  of  Selected  Region  of  a  Mammogram 
a.  Adaptive  background  trend  correction 

A  average  filter  of  kernel  size  of  23  x  23  pixels  is  used  to  remove  the  background  trend  in  a 
selected  region.  At  each  pixel  in  the  selected  region,  an  average  value  is  calculated  for  the 
surrounding  23  x  23  pixels  and  subtracted  from  the  original  pixel  value.  This  process  is 
repeated  for  all  the  pixels  in  the  selected  region.  The  adaptive  technique  is  used  to  isolate 
microcalcifications  from  their  immediate  surroundings  rather  than  from  a  different  region  that 
may  have  a  different  density  level.  As  long  as  the  sizes  of  microcalcifications  remain  small 
and  microcalcifications  are  not  clustered  too  close  to  each  other,  this  method  will  be  able  to 
preserve  the  original  shapes  of  microcalcifications.  The  kernel  size  of  23  pixels  is  equivalent 
of  approximately  1.2mm.  Therefore,  microcalcifications  that  are  larger  than  1.2mm  in  size 
are  eliminated  by  the  background  correction  process. 
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b.  Adaptive  thresholding 

After  background  correction,  a  threshold  is  applied  to  the  image  such  that  only  2048  pixels  of 
highest  intensity  (pixel  value)  are  kept  (assigned  pixel  value  1)  and  the  rest  pixels  are 
assigned  pixel  values  of  0.  The  number  of  pixels  that  are  kept  after  thresholding  is 
determined  empirically  that  the  sizes  of  microcalcifications  in  the  threshold  image  are  similar 
to  those  in  the  original  image. 

c.  Opening  operation 

A  morphological  Opening  operation  with  3x3  kernel  size  is  applied  to  the  binary  image  to 
remove  line  artifact.  The  Opening  is  a  combination  of  an  erosion  followed  by  a  dilation. 

d.  Dilation 

A  morphological  Dilation  operation  is  applied  after  the  OPEN  operation  to  fill  in  the  small 
holes  near  the  boundary  of  an  object. 

e.  AND  operation 

To  preserve  the  shape  of  microcalcifications,  an  AND  operation  between  the  original  binary 
image  and  the  image  after  dilation  is  performed. 

f.  Labeling 

A  labeling  process  is  applied  after  the  AND  operation  to  eliminate  small  objects  that  still 
remain  in  the  image  that  are  less  than  5  pixels  in  size.  Any  object  that  is  that  small  is 
unlikely  to  be  an  microcalcification. 

Image  Feature  Extraction 

Three  images  are  used  in  feature  extraction.  The  three  images  are:  Imagel,  the  preprocessed 
image;  Image2,  dilation  of  Imagel  by  a  5  x  5  kernel;  Image3,  dilation  of  Imagel  by  a  25  x  25 
kernel.  The  process  is  demonstrated  in  Figure  6. 

1 .  Number  of  microcalcifications  N  from  Imagel 

2.  Area  (number  of  pixels)  in  Imagel  S 1 

3.  Area  (number  of  pixels)  in  Image2  S2 

4.  Area  (number  of  pixels)  in  Image3  S3 

5.  (S2-S1)/N,  a  measure  of  average  number  of  irregular  pixels 

6.  (S3-S1)/N,  a  measure  of  average  distance  between  microcalcifications  in  a  cluster 

7.  (S2-S 1),  total  number  of  irregular  pixels 
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8.  (S3-S 1),  a  measure  of  average  distance  between  microcalcifications  in  a  cluster 

9.  VI,  size  of  the  largest  microcalcification  in  Imagel  -  size  of  average  microcalcification  in 
Image  1 

10.  V2,  size  of  the  average  microcalcification  in  Imagel  -  size  of  the  smallest 
microcalcification  in  Imagel 

11.  V3,  size  of  the  largest  microcalcification  in  Imagel 

12.  V4,  size  of  average  microcalcification  in  Imagel 

13.  V5,  size  of  smallest  microcalcification  in  Imagel 

14.  El,  ellipticity  of  the  largest  microcalcification  on  Imagel 

15.  WAE,  weighted  average  ellipticity  of  microcalcifications  on  Imagel 

16.  Ax,  length  of  semi-major  axes  of  the  fitted  ellipse  of  a  cluster  on  Imagel 

17.  Bx,  length  of  semi-minor  axes  of  the  fitted  ellipse  of  a  cluster  on  Imagel 

18. Ax/Bx 

19.  Ax  X  Bx 

A  total  of  19  image  features  are  extracted  for  candidate  cluster  and  used  as  input  to  a 
backpropagation  neural  network. 


(a)  (b)  (c)  (d)  (e) 

Figure  6  (a)  Original  gray  scale  image,  (b)  Binary  image,  (c)  Best-fit  ellipse  and  its  semi¬ 

major  axis  \  and  semi-minor  axis  B^^,  (d)  Dilation  with  5x5  kernel,  and  (e) 
Dilation  with  25  x  25  kernel. 

Genetic  Algorithms 

Genetic  algorithms  are  directly  modeled  after  biological  systems  and  behavior.  All  the  values 
of  a  structure  represent  the  neural  network  characteristics  that  uniquely  define  a  candidate 
solution  in  the  space  of  possible  solutions.  Utilizing  genetic  algorithms,  the  Genetic 
Supervisor  evolves  successor  populations,  or  generations,  from  a  limited  population  of  initial 
candidate  solutions.  It  does  this  by  treating  the  inclusion  or  exclusion  of  each  column  of  data 
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in  the  full  Input  column  set  as  features;  the  number  of  layers  and  the  number  of  neurons  per 
layers  as  features;  and  the  control  parameters  of  each  neural  network  as  features. 

These  features  are  then  varied  in  each  new  generation  with  the  resulting  structure  evaluated  in 
terms  of  neural  network  fitness.  Each  structure  in  the  generation  is  evaluated  and  judged  by 
either  the  lowest  RMS  Error  achieved  after  a  fixed  number  of  epochs  or  by  the  number  of 
epochs  taken  to  achieve  a  minimum  point  in  RMNS  Error.  These  two  measures  represent 
neural  networks  that  train  to  minimal  error  or  neural  networks  which  train  with  minimal 
epochs.  These  criteria  can  be  applied  to  the  set  of  training  cases  or  the  set  of  test  cases. 

If  the  structure  representing  a  neural  network  successfully  meets  the  fitness  criteria  selected, 
then  the  values  of  its  features  will  be  retained  and  bred  with  other  structures.  For  each 
generation,  the  Genetic  Supervisor  generates  a  population  of  structures  in  on  of  two  ways. 

All  the  structures  of  an  initial  generation  and  a  certain  number  of  structures  in  subsequent 
generations  are  created  with  features  set  to  random  values  constrained  within  specified  limits. 
Subsequent  generations  are  created  by  cross-breeding  the  strings  of  successful  structures  or 
occasional  mutations  of  randomly  selected  features  of  successful  structures.  Some  or  many 
of  the  weakest  structure  may  be  culled,  these  are  replaced  with  new  structures. 

Through  this  evolution-like  process,  an  optimal  neural  network  can  be  developed.  Note, 
however,  that  this  process  requires  the  training  of  many  versions  of  the  neural  network  to 
determine  an  optimal  one;  for  neural  network  models  Aat  have  large  network  configurations 
or  have  large  data  sets  this  can  be  a  lengthy  process  -  but  so  can  biological  evolution! 

A  system  consisting  of  a  Backpropagation  neural  network  and  a  Genetic  Supervisor  is 
developed  to  classify  the  extracted  image  features.  The  Genetic  Supervisor  attempts  to  select 
the  best  subset  of  image  features  from  the  provided  input  set,  configure  the  best  neural 
network  structure,  and  adjust  the  parameters  of  the  network  for  optimum  performance.  The 
performance  of  the  neural  network  system  is  evaluated  by  ROC  analysis. 

Reduction  of  Patient  Call  Backs 

A  third  set  of  images  will  be  selected  at  GUMC  which  are  images  of  patients  that  are  called 
back  after  screening  exam  to  follow  up.  A  majority  of  the  called  back  patients  are  benign. 

The  objective  is  to  reduce  the  number  of  call  back  patients  are  definitely  benign  and  therefore 
reduce  the  overall  cost  of  breast  cancer  diagnosis  practice. 

Database 

Pathoiooical  Specimen  Images 

Eighty  regions  of  interest  (ROI)  that  contain  clustered  microcalcifications  (40  benign  and  40 
malignant)  are  selected  from  60  digitized  radiographs  of  pathological  specimen  in  this  study. 
Figure  7  shows  all  of  the  80  ROIs  selected  in  the  database.  There  are  substantial  variations  in 
size  among  benign  or  malignant  microcalcifications.  The  information  concerning  the 
classification  of  microcalcifications  ("truth")  are  obtained  from  the  results  of  biopsy 
examination.  Background  trend  correction  is  employed  to  remove  the  non-uniform 
background  stmcture  in  different  ROIs. 
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Clinical  Mammograms 

Two  sets  of  clinical  mammograms  were  collected.  The  first  set  contains  70  cases  of  clusters 
of  microcalcifications  and  the  second  set  contains  62  cases  of  microcalcifications.  The  two 
sets  cases  were  selected  from  two  different  sources  of  medical  institutions.  The 
microcalcifications  in  both  sets  were  rated  subtle  by  radiologists.  The  diagnostic  truth  of  all 
of  the  cases  in  the  selected  sets  had  been  verified  by  biopsy. 
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(b) 


Figure  7  Database  for  the  training  and  testing  of  the  CNN;  (a)  40  ROIs  containing  benign 
clustered  microcalcifications  and  (b)  40  ROIs  containing  malignant  clustered 
microcalcifications. 
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Diagnosis  of  breast  cancer  by  MRI  using  a  neural  network  based  computer  vision 
system 

Figure  8  shows  the  overall  scheme  of  the  two-step  approach  that  is  employed  in  this  project. 
After  the  images  are  acquired,  an  interactive  image  display  and  image  analysis  is  used  to  aid 
the  radiologists’  examination  of  the  images  and  initial  diagnoses.  Meanwhile,  the  computer 
vision  system  will  detect  the  suspicious  areas  in  the  2D  slices.  The  detected  lesions  at 
various  slice  images  will  then  be  grouped  automatically  to  form  3D  lesions.  A  3D 
convolution  neural  network  will  be  employed  to  distinguish  the  3D  lesions  that  have  shapes 
typical  of  malignant  tumors  from  other  benign  lesions.  The  final  results  will  be  presented  to 
radiologists  to  make  a  final  diagnosis. 


Figure  8  Overall  scheme  of  the  neural  network  based  computer  vision  system 
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Image  acquisition  and  display 

The  BMR  examinations  are  performed  using  a  bilateral  dedicated  breast  coil  operating  in  the 
receive  mode  with  a  1.5  Tesla  MRI  system.  The  imaging  pulse  sequence  used  is  a  3D  FISP 
(fast  imaging  with  steady  state  free  precession)  variant,  a  rapid  gradient  echo  (GRE) 
technique  that  allows  volume  imaging  of  both  breasts  with  2.5mm  consecution  slice 
thickness.  Each  acquisition  sequence  uses  64  partitions  for  the  total  slab  thickness  of 
160mm.  The  matrix  size  of  2D  slice  field  of  view  is  256  x  256.  The  width  of  the  field  is 
approximately  380mm,  making  the  pixel  size  in  the  2D  slice  image  about  1.5mm.  The 
imaging  time  per  sequence  is  84  seconds. 

The  scans  are  performed  once  prior  to  contrast  administration  and  five  consecutive  times 
following  intravenous  gadolinium  bolus  administration.  No  delays  occur  between  each  of  the 
five  contrast-enhanced  scans.  The  images  are  calculated  after  all  five  enhanced  scans  are 
completed.  The  gadolinium  contrast  agent  is  administered  at  a  dose  of  0.1  mmol/kg  using 
one  of  the  three  FDA-approved  paramagnetic  agents:  gadolinium  diethylene  triamene 
pentaacetic  acid  demeglumine  (Gd-DTPA,  Berlex);  gadolinium  diethylene  triamene 
pentaacetic  acid  bis-methyl  amide  (Gd-DTPABMA,  Winthrop);  or  gadolinium  tris 
(hydroxymenthyl)  aminomethane  (Gd-D03A,  Squibb).  The  paramagnetic  agents  are  given 
via  the  largest  indwelling  catheter  cable  to  be  placed  in  a  peripheral  vein  over  five  seconds. 

The  contrast  injection  will  be  followed  by  lOcc  saline  flush  injection  to  clear  the  intravenous 
line  of  contrast. 

We  have  developed  an  image  registration  technique  to  match  image  slices  at  same  location 
during  a  time  sequence  so  that  the  pre-confrast  image  can  be  subtracted  correctly  from  the 
post-contrast  images.  Image  slices  are  stacked  up  to  construct  a  3D  image  object  by  volume 
rendering.  The  3D  object  can  be  rotated  and  viewed  from  different  angles.  The  user  can  also 
select  an  arbitrary  projection  plane  and  display  the  density  of  breasts  in  the  selected  plane. 

3D  images  in  one  complete  exam  before  and  after  contrast  injection  can  be  displayed  in  a 
continuous  loop  to  demonstrate  the  dynamic  change  of  signal  enhancement  in  the  lesions. 

The  3D  display  of  subtracted  images  after  contrast  injection  will  reveal  regions  of  strong 
signal  enhancement.  Radiologists  can  then  zoom  in  to  those  regions  of  strong  enhancement 
and  analyze  them  with  quantitative  image  analysis  tools  that  will  be  developed  in  this  system. 
The  image  analysis  tools  can  plot  enhancement  curves  at  any  given  location. 

Detection  of  suspicious  areas  in  images  of  2D  slice  images 

Enhancement  profiles  that  show  the  signal  intensities  at  different  time  intervals  before  and 
after  the  injection  of  contrast  agent  will  be  calculated  at  each  pixel  in  a  slice  image.  Features 
characterizing  the  signal  enhancement  after  contrast  injection  will  be  automatically  extracted 
from  the  enhancement  profiles.  The  features  will  be  calculated  based  on  time  derivatives  of 
the  enhancement  curve.  The  feature  selection  will  be  optimized  by  using  a  self-organizing 
feature  map  neural  network. 

A  three-layer,  feed-forward,  back-propagation  neural  network  will  then  be  employed  to 
recognize  pixels  with  abnormal  enhancement  profiles.  Enhancement  profiles  will  be  selected 
from  cancer  and  normal  regions  of  the  breasts  to  train  the  neural  network.  The  trained  neural 
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network,  provided  with  enhancement  profiles  of  individual  pixels  of  an  image,  will  be  able  to 
identify  those  pixels  belonging  to  a  region  that  could  be  potentially  cancerous.  A  2D  lesion  is 
formed  by  connecting  all  the  positive  pixels  identified  by  the  neural  network  in  a  local  area  in 
a  2D  slice  image.  The  detected  lesions  in  consecutive  2D  slice  images  will  be  grouped 
together  using  a  region  growing  technique  to  construct  3D  objects. 

Classification  of  3-D  suspicious  lesions 

Suspicious  regions  that  are  identified  in  the  previous  step  will  include  both  malignant  and 
benign  lesions.  Studies  have  shown  that  the  morphology  of  lesions  may  present  a  clue  to 
lesion  origin.57  The  presence  of  internal  septation  and  lobulation  suggest  the  lesion 
represents  a  fibroadenoma,  while  border  irregularity  and  rim  enhancement  are  highly 
suggestive  of  carcinoma.  The  morphology  of  lesions  is  better  delineated  in  3D  than  in  single 
2D  slice  images.  A  three-dimensional  convolution  neural  network  (CNN)  will  be  employed 
to  recognize  image  patterns  corresponding  to  benign  and  malignant  lesions.  CNN  has  been 
shown  to  be  an  effective  tool  for  two  dimensional  image  pattern  recognition.^*  We  have 
applied  CNN  in  many  different  applications  to  classify  various  image  patterns  in  chest  x-ray 
images  and  mammograms.  CNN  has  been  found  to  be  superior  to  the  conventional  fully 
connected  neural  networks  in  processing  two-dimensional  images  of  relatively  large  matrix 
sizes.^^ 

The  suspicious  areas  detected  in  a  slice  image  are  two-dimensional  projections  on  a  arbitrary 
plane  of  three-dimensional  lesions.  As  discussed  earlier,  the  morphology  of  lesions  may  be 
important  information  in  distinguishing  benign  lesions  from  malignant  cancers.  However, 
spiculated  boundaries  of  three  dimensional  lesions  may  not  show  up  in  some  of  their  two 
dimensional  projections.  As  a  result,  lesions  may  not  be  classified  correctly  based  on  their 
projections  on  a  2D  plane  alone.  To  improve  the  specificity  of  detecting  cancer  lesions, 
suspicious  areas  that  are  identified  in  each  of  2D  slice  images  will  be  grouped  together  to 
form  3D  lesions.  The  3D  lesions  will  be  classified  into  benign  and  malignant  types  of  lesions 
by  using  a  three  dimensional  convolution  neural  network.  The  classification  will  be  based  on 
the  three-dimensional  boundary  information  of  the  lesions. 

The  structure  of  the  3D  neural  network  is  demonstrated  in  Figure  2,  with  the  convolution  (the 
connections  between  the  input  layer  and  each  kernel  in  the  hidden  layer)  earned  out  in  three 
dimensional  space.  The  3D  convolution  neural  network  is  developed  based  on  the  2D  CNN 
model  that  has  been  successfully  applied  in  pattern  recognition  of  digital  manmiography.  In 
the  convolution  neural  network,  the  connections  between  two  layers  are  grouped  into  a 
number  of  clusters,  each  functioning  as  a  3D  convolution  filter.  Therefore,  the  processing  of 
input  signals  by  the  CNN  is  spatially  shift-invariant.  The  shift-invariance  makes  CNN 
suitable  for  image  pattern  recognition.  Pixel  values  of  the  3D  objects  will  be  used  as  input  to 
the  CNN. 
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Figure  9  Structure  of  a  3D  convolution  neural  network 


The  training  algorithm  of  the  3D  CNN53. 54  jg  similar  to  that  of  a  back-propagation  neural 
network.60  jhe  input  signals  are  now  three  dimensional  images.  The  weights  are  all 
arranged  in  the  convolution  kernels.  In  the  feed-forward  propagation,  the  outputs  of  /th  layer 
are  first  convoluted  with  weight  filters.  The  sum  of  the  convolution  is  then  added  by  a  bias 

term  to  form  the  net  input  to  the  next  layer. 
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where  N‘^\xyy,z)  is  the  net  input  to  the  unit  (x,  y,  z)  in  layer  l+l,  Opix,y,z)  is  the  output  of 
the  unit  in  layer  I,  Wp^(x,y,z)  is  a  weight  kernel,  and  is  a  bias  term  in  layer  /+/.  In  the 

notation,  layer  number  /  =  (1, 2, ...,  L),  cluster  number  in  the  /th  layer  p  =  (1, 2, ...,  P^),  and 
cluster  number  in  the  (/+i)th  layer  q  =  (1, 2, ...,  Note  that  *  denotes  discrete 

convolution. 
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In  the  error  back-propagation,  the  weights  are  modified,  similar  to  that  in  a  back-propagation 
neural  network,®®  as  the  following, 

AW;.,(n  +  1)  =  TKd‘;'*Ol)  +  aAW;.,(n),  [9] 


to  minimize  the  error  function. 
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E=\^iT{x,y,z)-0^{x,y,z)f ,  [10] 

where  T(x,y,z)  is  the  target  output. 
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RESULTS 

Images  of  pathological  Specimen 
Jackknife  Method 

A  jackknife  method  was  employed  to  evaluate  the  performance  of  the  CNN.  In  the  jackknife 
method,  half  of  the  ROIs  were  randomly  selected  from  the  database  of  80  ROIs.  These  ROIs 
were  used  to  train  the  convolution  neural  network.  The  other  half  of  the  ROIs  were  then  used 
to  test  the  performance  of  the  CNN.  By  choosing  different  random  samples  from  the 
database,  the  jackknife  test  can  be  repeated  to  generate  multiple  test  output  and  provide  a 
better  estimate  of  the  true  performance  of  the  CNN  in  classifying  benign  and  malignant 
clusters  of  microcalcifications. 

ROC  Analysis 

The  ou^ut  values  from  the  two  output  units  were  examined  by  using  Receiver  Operating 
Characteristic  (ROC)  analysis.^i.  62  The  LABROC4  algorithm's  developed  by  Metz  et  al. 
was  used  to  fit  ROC  curves  to  the  continuous  data  from  the  output  of  CNN.  The  area  under 
the  ROC  curve  (Az)  was  used  as  an  overall  measure  of  diagnostic  performance.  The  result 
from  each  jackknife  test  was  analyzed  individually  by  using  ROC  analysis.  Ten  jackknife 
tests  were  performed.  A  final  ROC  curve  was  obtained  by  averaging  the  results  from  the  10 
jackknife  tests,  as  shown  in  Figure  10.  The  CNN  system  performed  very  well  in  classifying 
benign  and  malignant  clusters  of  microcalcifications,  achieving  an  A^  value  of  0.90. 

Potential  Application  in  Recommending  Courses  of  Action 

A  potential  application  of  CNN  is  to  classify  microcalcifications  into  groups  of  definitely 
benign  and  possibly  malignant.  By  applying  a  low  threshold  level  to  the  output  values  of  the 
CNN,  we  can  make  CNN  a  classifier  that  is  not  very  specific  but  with  100%  sensitivity. 

With  such  a  classifier,  some  benign  microcalcifications  may  be  classified  as  possibly 
malignant,  but  all  of  the  microcalcifications  classified  as  benign  are  definitely  negative. 

Thus,  radiologists  can  ignore  the  clusters  of  microcalcifications  that  are  classified  as  benign 
and  only  concentrate  on  those  that  are  classified  as  possibly  malignant.  As  a  result,  the  time 
radiologists  spend  reading  mammograms  can  be  reduced  and  detection  efficacy  of  breast 
cancer  can  be  expected  to  improve. 
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Figure  10  ROC  analysis  of  the  performance  of  CNN  in  classifying  benign  and  malignant 
microcalcifications. 

The  ability  of  CNN  to  serve  as  a  classifier  to  eliminate  benign  microcalcifications  can  be 
demonstrated  in  Table  I.  In  each  of  the  jackknife  tests,  we  set  threshold  levels  of  the  output 
of  CNN  such  that  all  of  the  malignant  ROIs  are  to  be  called  positive  by  the  computer  system 
and  calculate  the  number  of  the  benign  ROIs  that  can  be  called  negative  (i.e.,  have  output 
values  below  the  threshold  level  for  positive  ROIs). 

Table  I  shows  the  results  for  each  individual  jackknife  test  as  well  as  the  averaged  results  of 
the  10  jackknife  tests  (second  column).  The  CNN  can  identify,  on  average,  approximately 
42%  of  the  benign  clusters  of  microcalcifications  with  100%  sensitivity  (without  missing  any 
malignant  clusters).  Therefore,  if  this  CNN  system  were  used  to  help  radiologists  in 
detecting  malignant  microcalcifications,  radiologists  would  only  need  to  examine  about  half 
of  the  detected  microcalcifications. 

As  discussed  earlier,  studies  have  shown  that  approximately  10-30%  of  breast  cancers  are 
missed  by  mammography  and  only  10-30%  of  biopsy  cases  recommended  by  mammography 
are  actually  malignant.  As  a  comparison  to  the  reported  performance  of  radiologists  in  breast 
cancer  diagnosis,  we  also  listed  in  Table  I  (third  and  fourth  columns)  the  average  specificities 
and  positive  predictive  values,  defined  as  the  portion  of  malignant  cases  among  the  biopsied 
cases. 

With  a  sensitivity  of  80%,  approximately  the  same  level  of  sensitivity  reported  by  average 
radiologists,  the  neural  network  system  achieved  a  positive  predictive  value  for  malignancy 
of  84%,  compared  with  10-30%  achieved  by  radiologists.  Therefore,  the  CNN  appears  to  be 
a  very  promising  tool  for  assisting  radiologists  in  making  decisions  for  the  diagnosis  of  breast 
cancer. 
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Real  mammograms 

Genetic  algorithms  are  employed  to  optimize  the  structure  of  a  neural  network  and  select  the 
best  performing  subset  from  the  initial  image  features.  Additional  cases  of  patients  that  have 
been  recalled  for  further  examination  are  being  collected  at  GUMC.  The  performance  of  the 
neural  network  system  based  on  these  cases  will  help  determining  the  usefulness  of  the 
system  in  reducing  unnecessary  biopsies. 

Optimized  Feature  Set  bv  Genetic  Algorithms 

Genetic  Algorithms  were  applied  to  the  first  subset  of  seventy  cases  selected  from  the 
database.  Two  different  sets  of  image  features,  each  consisting  of  six  features,  were  selected 
as  a  result  of  the  optimization  process.  The  neural  network  performed  equally  well  with  the 
two  selected  subsets. 

Feature  Set#1: 

1 .  V3,  size  of  the  largest  microcalcification  in  hnagel 

2.  V4,  size  of  average  microcalcification  in  hnagel 

3.  V5,  size  of  smallest  microcalcification  in  hnagel 

4.  El ,  ellipticity  of  the  largest  microcalcification  on  hnagel 

5.  WAE,  weighted  average  ellipticity  of  microcalcifications  on  hnagel 

6.  Ax,  length  of  semi-major  axes  of  the  fitted  ellipse  of  a  cluster  on  hnagel 

Feature  Set  #2: 

1 .  El ,  ellipticity  of  the  largest  microcalcification  on  hnagel 

2.  WAE,  weighted  average  ellipticity  of  microcalcifications  on  hnagel 

3.  Ax,  length  of  semi-major  axes  of  the  fitted  ellipse  of  a  cluster  on  hnagel 

4.  Bx,  length  of  semi-minor  axes  of  the  fitted  ellipse  of  a  cluster  on  hnagel 

5.  V3-V4 

6.  V4-V5 

The  optimized  parameters  for  the  neural  network  are:  learning  rate  0.7  ~  0.95  and  momentum 
0.02-0.06. 


DAMD17-94-J-4377 


—  27  — 


Final  Report 


Diagnosis  of  Breast  Cancer 


Chris  Y.Wu,Ph.D. 


Figure  1 1  Performance  of  feature-based  neural  network  in  classification  of 

microcalcifications 

We  evaluated  the  performance  of  the  optimized  neural  network  by  using  round-robin  method 
and  ROC  analysis.  Figure  1 1  shows  the  ROC  curve  for  the  classification  of 
microcalcifications  based  on  extracted  image  features. 

The  system  of  neural  network  and  genetic  algorithms  improved  performance  over  our 
previous  TRBF  neural  network^^.  The  neural  network  system  was  able  to  classify  benign  and 
malignant  microcalcifications  at  a  level  favorably  compared  to  experienced  radiologists.  On 
the  same  database  evaluated,  the  ROC  curve  for  an  experienced  radiologist  yielded  an  Az  of 
0.54,  while  the  Az  of  the  ROC  curve  for  the  neural  network  with  optimized  stracture  is  0.68. 
The  use  of  the  neural  network  system  can  be  used  to  help  radiologists  reducing  the  number 
biopsies  in  clinical  applications. 

Case  demonstration  of  3D  image  visualization 

Figure  12  demonstrates  a  display  of  selected  2D  slice  images  (a),  a  constructed  3D  image  (c), 
and  signal  enhancement  curve  of  a  selected  region  (d).  The  enhancement  curves  are  used  by 
radiologists  to  determine  the  probability  of  malignancy  of  a  specific  region.  Other 
characteristics  of  signal  enhancement  such  as  enhance  and  washout  time,  gradients  of 
enhancement  curves,  and  variations  among  a  selected  region  can  also  be  calculated. 
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Time 


(d)  Enhancement  curve  of  a  selected  region 
Figure  12  Image  visualization  in  BMRI 

Two  patient  exams  are  demonstrated  here,  each  exam,  the  64  two-dimensional  slices  at 
each  time  interval  are  stacked  up  to  construct  a  three  dimensional  image.  Volume  rendering 
as  well  as  surface-rendering  are  used  to  visualize  the  3D  image  object.  For  each  patient 
exam,  there  are  one  pre-contrast  3D  image  and  5  consecutive  post-contrast  3D  images.  The 
six  reconstructed  3D  images  are  displayed  in  a  continues  loop  so  that  the  change  of  signal 
enhancement  over  time  can  be  observed.  In  each  of  the  3D  images,  a  2D  projection  plane  can 
be  selected  at  any  orientation  to  step  through  the  3D  image  and  display  the  signal  intensity 
within  the  2D  plane.  The  user  can  also  point  to  any  specific  location  in  a  2D  image  and 
display  the  pixel  value  at  that  location.  A  user  can  also  select  a  specific  region  in  any  2D 
plane  and  display  the  change  of  signal  intensity  over  the  time  sequence  to  obtain  signal 
enhancement  curves. 

Li  the  first  case  demonstrated  here,  a  patient  has  a  tumor  in  the  right  breast  which  is  almost 
invisible  in  the  original  images.  It  can  be  identified  on  the  subtracted  images,  as  shown  in 
Figure  13. 

The  second  case  is  a  normal  case.  The  image  subtraction  and  3D  reconstruction  make  it 
easier  for  radiologists  to  locate  potential  tumor.  The  radiologists  can  examine  the  suspicious 
area  in  detail  from  the  2D  projections  planes  in  which  the  suspicious  areas  are  located. 

Figure  14  shows  the  signal  eidiancement  of  subtracted  2D  slice  images. 
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Figure  13  Effect  of  image  subtraction:  (a)  3D  original  image  (b)  3D  subtracted  image  (c)  2D 

original  slice  (d)  2D  subtracted  slice 


(a) 


(b) 


(c) 


(A) 


(e) 


(f) 


Figure  14  Signal  enhancement  of  breast  images  in  MRI:  (a)  pre-contrast  image  (b-f)  5  post¬ 
contrast  images. 
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DISCUSSIONS 

It  is  important  to  note  that  the  CNN  is  designed  as  an  automated  classifier  of 
microcalcifications  for  the  diagnosis  of  breast  cancer.  It  will  be  used  in  conjunction  with 
other  schemes  for  the  detection  of  microcalcifications  in  digital  mammograms.  Once 
microcalcifications  are  detected,  the  CNN  will  be  applied  to  classify  them  into  benign 
(negative)  and  malignant  (positive)  groups.  Radiologists  can  ignore  the  microcalcifications 
that  are  classified  into  the  benign  group  and  examine  those  that  are  classified  as  malignant  to 
decide  whether  to  recommend  biopsy  or  short  term  follow-up  exams. 

Table  I.  Application  of  CNN  in  Recommending  Courses  of  Action 


Jackknife  Test  Conectly  identified  False  Positives  at  False  Positives  at 

negative  cases  at  90%  sensitivity  80%  sensitivity 


100%  sensitivity 


*  Positive  Predictive  Value — Defined  as  the  portion  of  the  actually  positive  cases  among  the 
cases  diagnosed  that  are  classified  as  positive  by  a  diagnostic  system. 

The  results  discussed  are  based  on  radiographs  of  biopsy  specimen  of  microcalcifications. 
The  specimen  images  have,  in  general,  better  image  qu^ity  and  ^eater  signal-to-noise  ratio 
than  the  regular  mammograms.  The  radiographs  digitized  with  high  resolution  digitizers 
provide  the  morphological  information  of  individual  microcalcifications  that  makes  the 
classification  of  microcalcifications  into  benign  and  malignant  groups  possible.  The  CNN 
system  will  need  to  be  tested  on  regular  mammograms.  Some  parameters  of  CNN  may  need 
to  be  fine  tuned  when  applied  to  regular  mammograms  and  the  CNN  system  may  not  achieve 
the  same  performance  level  as  it  did  in  this  study. 

Both  a  large  training  and  testing  database  are  necessary  in  order  to  train  and  evaluate  the 
performance  of  the  neural  network  sufficiently  and  reliably.  We  will  be  expanding  our 

significantly  in  the  future.  To  further  improve  the  accuracy  of  the  classification,  a 
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hybrid  neural  network  (HNN)65  will  also  be  employed  to  classify  microcalcifications  based 
on  the  input  of  both  image  data  and  image  features^^  that  will  be  automatically  extracted. 

The  evaluation  of  feature  based  neural  network  with  Generic  Algorithm  was  based  on  a  very 
difficult  data  set  which  was  different  from  what  was  used  for  evaluation  of  image  based 
neural  networks.  The  microcalcification  clusters  in  the  data  set  for  feature  based  neural 
network  are  very  subtle  and  an  experienced  radiologist  only  scored  an  Az  of  0.54  in 
classifying  between  benign  and  malignant  clusters.  The  size  of  the  database  is  also  small. 
We  are  currently  in  the  process  of  expanding  our  database  to  include  more  mammograms 
from  different  hospitals. 
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Conclusions 

We  have  demonstrated  that  the  convolution  neural  networks  can  be  an  effective  tool  in  the 
diagnosis  of  breast  cancer.  The  results  obtained  in  this  study  are  very  promising,  even  though 
they  were  based  on  a  relatively  small  training  and  testing  database.  These  results  indicate  the 
potential  usefulness  of  CNN  in  classification  of  microcalcifications  in  digital  manunograms. 
An  extensive  clinical  test  of  our  developed  system  using  real  mammograms  wiU  be  needed  to 
determine  the  clinical  applicability. 

Genetic  algorithms  are  an  effective  tool  to  select  optimal  input  features  and  structure  of  a 
backpropagation  neural  network.  The  neural  network,  combined  with  genetic  algorithms,  is 
able  to  effectively  classify  benign  and  malignant  microcalcifications.  The  results  of  the 
neural  network  system  can  be  used  to  help  reducing  the  number  of  benign  biopsies. 

We  have  developed  an  image  visualization  system  to  aid  radiologists  to  analyze  breast  MRI 
images  efficiently  and  accurately.  The  interactive  image  visualization  and  quantitative  image 
analysis  enable  radiologists  to  examine  breast  MRI  cases  more  efficiently  and  accurately.  We 
are  dso  developing  an  automated  detection  system  that  can  locate  the  potential  tumors  from 
the  acquired  images  based  on  signal  enhancement  pattern  and  intensity.  This  visualization 
system  can  make  BMRI  more  accurate  and  cost-effective,  and  thus  make  BMRI  clinically 
feasible. 
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